The EVIDEM framework and its usefulness for priority setting across a broad range of health interventions

Commentary This commentary responds to the article by Goetghebeur et al [1], which applies the EVIDEM (Evidence and Value: Impact on DEcision-Making) framework to evaluate growth hormone therapy for Turner syndrome patients. The EVIDEM framework is developed to assist decision-makers in healthcare decisions, and encompasses a multi-criteria decision analysis (MCDA) matrix consisting of 15 quantifiable, and six qualitative components of decision. With this comprehensive set of criteria, relevant experts can assess the performance of health interventions, and results are input for informed and transparent healthcare decisions. Goetghebeur et al [1] propose that the EVIDEM can be used to compare various interventions across disease areas in order to prioritize interventions. They suggest that the EVIDEM can analyze single interventions, and the performance of competing interventions can subsequently be compared in a performance matrix. While we value the qualities of the EVIDEM because of its scope and breadth, we have doubts on the results consistency of the EVIDEM to compare competing interventions, particularly when setting priorities across broad healthcare service areas (e.g. in designing the national health benefit package) for two main reasons. First, the EVIDEM framework ignores the contextual nature of priority setting process by assuming a set of universal priority setting criteria [2]. In reality, the priority setting process is context specific and different sets of criteria lead priority setting of health interventions in different contexts. As the examples of studies in Nepal [3], Chile [4], and Ghana [5] show, the set of criteria identified for using in priority setting of health interventions were different between countries. Therefore, we suggest that the setting of prioritization criteria needs to be locally determined or verified, implying that the EVIDEM needs to be flexible to allow change/modification of the components to suit the local context. Secondly, the EVIDEM is vulnerable to interventions ranking inconsistency where performance evaluation of a broad range of competing interventions is mandated. For example, the EVIDEM framework requires different expert panels to assess the performance of every single intervention separately. This may lead to inconsistency of the results as different expert panels may have different considerations across the broad range of interventions to be assessed. As shown in the EVIDEM Turner Syndrome case study [1], the panel of experts estimated growth hormone intervention to achieve 41% of maximum value. However, in the absence of established explicit weights of criteria, it is not certain that the same panel will be consistent in evaluating different interventions, or that another panel of experts comes up with the same or similar value. These arguments raise the question whether the approach of EVIDEM is locally meaningful and consistent when priorities are set for a range of interventions. To address its limitations, we propose a stepwise process to identify criteria and their weights, and rank ordered interventions. We suggest that, to set priorities of a range of interventions within a certain context, a locally-meaningful set of criteria and their relative importance (i.e. weights) should be elicited by consulting relevant stakeholders. As a next step, the identified set of criteria and weights are then used to consistently assess the performance of the broad range of interventions. To illustrate the method, we describe a study in Thailand [6], which defined explicit criteria to prioritize health interventions for the national health benefit * Correspondence: [email protected] † Contributed equally Nijmegen International Center for Health Systems Research and Education (NICHE), Department of Primary and Community Care, Radboud University Nijmegen Medical Centre, Nijmegen, The Netherlands Full list of author information is available at the end of the article Youngkong et al. Cost Effectiveness and Resource Allocation 2011, 9:8 http://www.resource-allocation.com/content/9/1/8